1. Field of the Invention
This invention relates to image analysis, specifically to identifying friendly and enemy units in a soldier's head-mounted display.
2. Discussion of Prior Art
In September 2000, the U.S. Army performed the first operational test of the Land Warrior soldier system. The system connects soldiers on the battlefield via a wireless network. In addition, each Land Warrior carries a computer connected to a GPS receiver, scope video camera, and head-mounted display. The head-mounted display presents video from the scope camera attached to the soldier's weapon. As a result, the soldier can point his or her gun around a corner to assess enemy activity while minimizing exposure to attack.
A Land Warrior's head-mounted display serves as a targeting system. A soldier aims his or her weapon by placing a crosshair over an enemy unit in the scope video. In the heat of battle, however, it is difficult to quickly determine whether an individual is a friendly or enemy unit. As a result, friendly fire occurs in which a soldier accidentally fires on a fellow soldier.
To combat friendly fire, the present inventor developed a system that differentiates friend from foe in a soldier's head-mounted display. The system leverages position information reported by friendly soldiers. In addition, if aerial imagery of the battlefield is available, the system enables commanders to tag unknown individuals as friend, foe, or neutral.
Existing systems served as a foundation for the present invention. These existing systems are described below.
Tracking Objects in Video
The present invention relies on an object tracker to detect units in aerial imagery and scope video. Object trackers perform image analysis to locate, recognize, and follow objects in video sequences. Inventors have devised several suitable object trackers. For example:                U.S. Pat. No. 6,529,613 to Astle (2003) relies on image templates to determine the positions of objects in video.        U.S. Pat. No. 6,507,660 to Wirtz et al. (2003) uses a reference image and performs edge detection to identify objects in aerial imagery.        U.S. Pat. No. 6,542,621 to Brill et al. (2003) performs probabilistic analysis to deal with occlusion while tracking multiple objects in video sequences.        U.S. Pat. No. 6,434,254 to Wixson (2002) detects objects in video images using one method suitable for daytime scenes and a second method suitable for nighttime scenes.        
The present invention builds on these object trackers by associating objects detected in aerial imagery with those detected in scope video. As a result, when a commander identifies an enemy unit in the aerial imagery, the present invention labels the corresponding unit in a soldier's scope video.
Determining an Object's Position
The present invention determines the position of unknown units in aerial imagery and uses this information to identify unknown units that appear in a soldier's scope video. Several existing systems use multiple cameras to determine the position of objects. For example:                U.S. Pat. No. 6,539,330 to Wakashiro (2003) determines the three-dimensional position of objects based on images from two cameras.        
The present invention takes a different approach in determining the position of unknown units. The battlefield's topology restricts one dimension of movement for ground troops. For the remaining two dimensions, the present invention performs a calculation based on the unknown unit's position in an aerial image and the image's latitude and longitude extents. This technique enables the present invention to deduce the position of an unknown unit using a single camera.
Labeling Objects in Video
The present invention labels friendly and enemy units that appear in a soldier's head-mounted display. Similarly, several existing systems identify objects such as waypoints that appear in real-world images. For example:                U.S. Pat. No. 5,786,849 to Lynde (1998) identifies waypoints, obstacles, and targets that appear in marine navigation binoculars.        
The present invention combines information about the position and orientation of a soldier's scope video camera with knowledge friendly and enemy unit locations to accurately label objects in a soldier's head-mounted display.
Differentiating Friend from Foe
In response to friendly fire incidents during the Gulf War, the U.S. Army launched a combat identification program. The program's goal was to equip soldiers with systems for differentiating friend from foe in combat. The result was an interrogator/transponder pair carried by each friendly unit. To determine whether a unit was a friend or foe, a soldier aimed his or her interrogator at the unit. For units carrying a properly configured transponder, the interrogator would indicate that the individual was a friend. Otherwise, the system deduced that the unit was an enemy.
The present invention has several advantages over combat identification systems that employ an interrogator and transponder. First, the present invention requires no equipment in addition to that already worn by Land Warriors. Second, the present invention requires to effort on the part of soldiers to differentiate friend from foe. The present invention automatically identifies units that appear in a soldier's scope video. As a result, the present invention enables soldiers to make rapid firing decisions.